Control  of  Mobile  Communication  Systems  with 
Time- Varying  Channels  via  Stability  Methods 


Robert  Buche* 
Mathematics  Dept. 

North  Carolina  State  University 
Rayleigh,  NC  27695 
rtbuche@unity.ncsu.edu 


Harold  J.  Kushner 
Applied  Mathematics  Department 
Lefschetz  Center  for  Dynamical  Systems 
Brown  University,  Providence,  RI  02912 
hjk@dam.brown.edu 

December,  2002;  Revised  August  2003  and  March  2004 


Abstract 

Consider  the  forward  link  of  a  mobile  communications  system  with  a 
single  transmitter  and  connecting  to  K  destinations  via  randomly  varying 
channels.  Data  arrives  in  some  random  way  and  is  queued  according  to  the 
K  destinations  until  transmitted.  Time  is  divided  into  small  scheduling 
intervals.  Current  systems  can  estimate  the  channel  (e.g,  via  pilot  signals) 
and  use  this  information  for  scheduling.  The  issues  are  the  allocation  of 
transmitter  power  and/or  time  and  bandwidth  to  the  various  queues  in 
a  queue  and  channel-state  dependent  way  to  assure  stability  and  good 
operation.  The  decisions  are  made  at  the  beginning  of  the  scheduling 
intervals.  Stochastic  stability  methods  are  used  both  to  assure  that  the 
system  is  stable  and  to  get  appropriate  allocations,  under  very  weak  con¬ 
ditions.  The  choice  of  Liapunov  function  allows  a  choice  of  the  effective 
performance  criteria.  The  resulting  controls  are  readily  implementable 
and  allow  a  range  of  tradeoffs  between  current  rates  and  queue  lengths. 

The  various  extensions  allow  a  large  variety  of  schemes  of  current  interest 
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author  would  like  to  thank  the  Dept,  for  the  incredibly  supportive  research  environment  that 
it  provided. 
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to  be  covered.  All  essential  factors  are  incorporated  into  a  “mean  rate” 
function,  so  that  the  results  cover  many  different  systems.  Because  of  the 
non-Markovian  nature  of  the  problem,  we  use  the  perturbed  Stochastic 
Liapunov  function  method,  which  is  well  adapted  to  such  problems.  The 
method  is  simple  and  effective. 


1  Introduction 

We  consider  the  problem  of  power  and  time  control  for  the  forward  link  of  a 
mobile  communications  system  when  the  connecting  channels  are  randomly  time 
varying.  The  control  can  be  over  allocated  time,  power,  and  bandwidth,  or  over 
all  simultaneously.  More  discussion  of  the  possibilities  appears  in  Section  4. 
There  are  K  queues  at  the  base  station,  each  receiving  data  according  to  some 
random  process,  and  each  associated  with  a  unique  mobile  destination.  The 
(nonnegative)  content  of  any  queue  is  simply  its  initial  content  plus  its  arrivals 
minus  its  departures  to  date.  Time  is  divided  into  small  scheduling  intervals 
of  length  A  (typically,  in  the  low  tens  of  milliseconds)  and  the  transmitter 
decisions  concerning  the  possible  power,  time,  and  bandwidth  allocations  are 
made  at  the  beginning  of  the  intervals.  With  the  appropriate  use  of  pilot  signals, 
many  systems  have  the  ability  to  estimate  key  properties  of  the  channel  (e.g., 
S/N  ratios)  and  to  use  this  information  to  determine  the  controls.  Such  an 
approach  would  greatly  improve  the  performance  [1].  More  generally,  there 
might  be  only  partial  knowledge  of  the  channel.  The  actual  physical  situation 
that  corresponds  to  a  channel  state  value  or  to  an  estimate  of  the  value  is 
unimportant.  Each  value  of  the  state  or  of  its  estimate  corresponds  to  some  set 
of  allowed  possibilities  for  the  coding  and  resource  allocations.  For  simplicity,  it 
is  assumed  that  the  controller  quantizes  the  state  of  the  K  channels  into  a  finite 
number  of  values,  indexed  by  (a  A^-vector)  j:  each  value  of  j  denotes  the  state 
of  all  of  the  individual  channels.1  The  individual  components  of  the  channel 
state  might  or  might  not  be  mutually  independent.  The  rates  of  transmission 
are  measured  in  terms  of  packets.  The  queue  state  and  data  arrival  processes 
need  not  be  Markovian.  The  actual  transmission  schemes  that  are  allowed  are 
quite  general.  They  can  be  based  on  TDMA,  CDMA,  bit  interval  control,  or  on 
various  combinations. 

To  best  accommodate  issues  of  fairness  as  well  as  the  particular  performance 
criteria  of  interest,  the  power  and  time  allocation  in  each  scheduling  interval 
should  depend  on  the  lengths  of  the  queues  as  well  as  on  the  current  knowl¬ 
edge  of  the  (vector- valued)  state  of  the  channel.  The  resulting  optimal  control 
problem  is  quite  difficult.  The  dimension  (K)  might  be  high,  the  queue-length 
processes  might  not  be  Markovian  under  any  control  scheme,  and  uncertainty  of 
the  channel  state  can  lead  to  a  complicated  filtering  problem.  These  complica¬ 
tions  suggest  an  approximation  approach.  The  small  duration  of  the  scheduling 
interval  A  and  the  (usually)  fast  rate  of  change  of  the  channel-state  process 
suggest  an  approach  to  the  control  problem  that  is  based  on  a  “crude  mean 

1The  finiteness  of  the  number  of  channel  states  is  assumed  only  for  notational  simplicity. 
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flow”  idea.  Loosely  speaking,  we  require  only  that  there  is  some  allocation  un¬ 
der  which  an  appropriately  defined  “mean”  system  is  stable  (stability  is  defined 
in  the  next  section),  and  then  the  stability  approach  can  be  carried  out  under 
quite  general  conditions  for  a  great  variety  of  physical  systems. 

The  conditions  of  typical  applications  (e.g.,  Rayleigh  fading)  do  not  usually 
allow  a  Markov  representation  of  the  queue-length  and  channel-state  processes, 
even  under  “feedback”  control;  hence  the  usual  methods  of  stability  analysis, 
which  require  a  Markov  model,  are  inappropriate.  Because  of  the  non-Markov 
and  randomly-time-varying  nature  of  the  problem,  the  stability  is  proved  using 
the  perturbed  Liapunov  function  methods  of  [8].  With  this  method,  one  starts 
with  a  basic  Liapunov  function  that  works  for  a  simpler  (e.g.,  a  “crude  mean 
flow”)  system.  Then  one  obtains  finds  a  perturbation  to  this  basic  Liapunov 
function  which  can  be  used  as  a  Liapunov  function  for  the  actual  non-Markov 
physical  system.  Analogously  to  the  usual  “stability  method”  procedure  that  is 
used  to  get  controls,  the  controls  are  determined  by  maximizing  the  (conditional 
expectation,  given  the  current  data,  of  the)  negative  of  the  gradient  of  the  basic 
(not  the  perturbed)  Liapunov  function.  The  development  uses  a  basic  Liapunov 
function  that  is  a  polynomial  which  is  the  sum  of  terms,  each  depending  on  only 
one  component  of  the  state  of  the  queue.  This  is  adequate  for  applications  at 
this  time.  Liapunov  functions  of  a  more  general  form  can  also  be  used,  the  main 
requirement  being  that  the  gradient  has  positive  components.2 

Stability  is  of  interest  partly  because  it  assures  a  robustness  of  behavior  to 
“small”  changes  in  the  system.  For  this  reason  an  analysis  based  on  Markov 
models  is  inadequate.  No  physical  model  is  truly  Markov.  If  an  approximating 
Markov  model  is  used  for  analysis,  then  it  is  important  to  know  that  the  stability 
holds  for  the  original  systems  that  are  approximated.  The  perturbed  Liapunov 
function  method  is  a  powerful  tool  for  checking  such  robustness  since  it  does  not 
require  Markovianness.  Additionally,  mild  modifications  of  the  proof  allow  the 
basic  data  (e.g.,  the  probabilistic  structure  of  the  data  arrival  process)  to  be  time 
varying.  The  Liapunov  function  can  be  chosen  to  reflect  the  relative  importance 
of  the  various  queues  or  even  to  accommodate  constraints.  For  example,  if  it 
is  desired  to  minimize  the  time  that  queue  i  is  above  a  value  A,;,  one  can  let 
the  dependence  of  the  Liapunov  function  on  the  state  of  queue  i  be  very  large 
when  it  exceeds  Ai.  Analogously  to  the  treatment  of  a  related  problem  in  [5], 
where  a  sequence  of  systems  scaled  by  speed  was  considered,  one  can  allow  a 
received  packet  to  be  rejected  by  the  receiver  if  it  decides  that  it  contains  too 
many  errors.  Then  the  acceptance  probability  will  also  depend  on  the  allocated 
power. 

The  assumptions  and  the  precise  definition  of  stability  are  in  Section  2, 
where  the  assumptions  are  discussed  and  seen  to  be  unrestrictive,  and  a  simple 
example  given.  The  main  theorem  is  stated  and  proved  in  Section  3.  The  scope 
of  the  approach  and  simplicity  of  application  can  be  seen  from  the  examples 
in  Sections  4  and  5,  which  can  be  read  without  knowledge  of  Section  3.  Some 

2For  example,  if  we  wish  to  more  or  less  equalize  the  lengths  of  the  various  queues,  one 
can  add  terms  to  the  Liapunov  function  that  depend  on  the  differences  of  the  values  of  the 
queue  components. 
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canonical  classes  of  physical  models  (and  various  combinations  and  extensions) 
are  discussed  in  Section  4.  These  include  the  standard  CDMA  and  TDMA  and 
show  that  many  types  of  constraints  can  be  handled.  The  approach  and  results 
are  the  same  when  the  channel  is  only  partly  known  or  retransmission  of  poorly 
received  packets  is  called  for.  A  similar  analysis  can  be  used  when  there  are 
multiple  antennas  [15]  and  frequencies.  Multiple  antennas  and  frequencies  are 
used  in  the  space-time  coded  OFDM  (orthogonal  frequency  division  multiplex¬ 
ing)  approach  [10,  13],  part  of  the  purpose  of  which  is  to  provide  “space-time 
diversity,”  to  partially  neutralize  the  effects  of  channel  variations.  But,  via  the 
approach  of  this  paper  and  given  sufficient  information  on  the  channel,  one 
might  sometimes  find  it  preferable  to  use  only  the  most  appropriate  combina¬ 
tions  of  antennas  and  frequencies  for  the  various  sources  at  any  time.  Section 
5  contains  a  comparison  of  the  obtained  controls  with  optimal  controls  for  a 
closely  related  problem. 

For  the  special  case  of  problems  of  the  type  of  Example  4.2  where  the  rate  of 
transmission  is  proportional  to  power,  [1,  11]  obtain  rules  for  power  allocation 
whose  form  is  similar  to  ours  and  which  are  based  on  stability  considerations,  al¬ 
though  the  method  uses  large  deviations  estimates  and  the  setup  is  Markovian. 
The  reference  [14]  was  perhaps  the  first  to  consider  the  problem  of  dynamic 
power  allocation  when  the  channels  are  time-varying.  But,  since  their  channel- 
rate  and  data-arrival  processes  are  all  i.i.cl.  sequences,  the  range  of  applications 
is  very  small.  The  reports  [2,  3]  deal  with  related  problems.  They  prove  in¬ 
teresting  results  concerning  the  limit  (as  t  — >  oo)  of  (queue  length  at  f)/f,  and 
show  that  (under  appropriate  conditions)  this  limit  is  zero.  This  is  used  to 
show  that  the  integral  of  the  “rates”  of  transmission  per  unit  time  converges. 
But  such  a  result  does  not  imply  stability  of  the  queue  length  process,  since  it 
can  grow  sublinearly.  [Consider,  for  example,  a  process  modeled  by  a  random 
walk  in  one  dimension,  with  reflection  at  the  origin,  where  the  queue  is  only 
null  recurrent.]  Their  scheduling  is  done  continuously,  rather  than  discretely  in 
time.  They  allocate  a  single  resource  (e.g.,  bandwidth)  and  the  rate  is  propor¬ 
tional  to  the  allocation.  This  focus  implies  a  more  restricted  set  of  applications. 
For  example,  it  does  not  seem  to  be  able  to  handle  the  “discrete  selection”  bit 
interval  controls,  the  multiple  antenna  problem,  or  the  joint  bandwidth-power 
allocation  problem,  or  nonlinear  dependencies  on  the  controls  in  general  (as  in 
Example  4.1).  Their  setup  and  proofs  are  much  more  complicated  than  that 
here.  Stability  of  the  “rate  of  transmission”  process  in  our  case  follows  from 
Theorem  3.1. 

The  work  that  is  closest  to  ours  is  [12],  where  the  channel-state  process  is 
Markov,  the  data  input  sequence  is  i.i.d.,  and  a  “complete  resource  pooling” 
condition  is  required.  The  decision  rule  is  the  same  as  ours  for  a  quadratic 
Liapunov  function.  The  emphasis  is  on  stability  in  the  heavy  traffic  limit,  and 
showing  how  the  problem  simplifies  there.  Stability  in  the  heavy  traffic  limit 
does  not  imply  stability  for  any  one  of  the  “prelimit”  processes.  See  also  [5]  for 
a  stability  analysis  as  the  heavy  traffic  regime  is  approached. 
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2  Assumptions  and  the  Control  Rule 


Comment  on  controls.  For  purposes  of  concrete  visualization,  one  could 
suppose  that  the  controls  are  over  either  the  power  and/or  time  or  bandwidth 
allocated  to  each  queue.  Other  forms  are  noted  in  the  examples  in  Section  4.  Let 
ut(j,  x)  denote  the  control  applied  to  queue  i  in  a  scheduling  interval,  when  the 
(vector-valued)  queue  state  is  x  and  the  estimate  of  the  (vector-valued)  channel 
state  is  j.  Some  component  of  the  control  might  also  be  a  probability;  e.g.,  as 
in  the  example  at  the  end  of  the  section,  where  the  control  is  the  probability  of 
selecting  a  queue  in  an  interval.  The  constraints  need  to  be  taken  into  account  in 
defining  admissible  controls.  For  example,  the  total  time  scheduled  in  an  interval 
cannot  be  greater  than  A.  In  the  simplest  model,  where  Ui(j,x)  denotes  power, 
the  total  allocated  power  cannot  be  greater  than  that  available  and  then  we 
have 

^2ui{j,x)  <  p,  (2.1) 

i 

where  /!  is  the  (real-valued)  total  available  power.  More  generally,  one  might 
only  require  that  the  power  “  average  locally  in  time”  to  that  available.  Other 
possible  constraints  might  be  a  maximum  length  for  any  queue,  or  the  control 
might  have  to  take  values  in  some  discrete  set.  Obviously,  the  queue  contents 
Xi  must  be  nonnegative. 

Assumption  (A2.1)  gives  the  general  definition  of  admissibility  and  gener¬ 
alizes  (2.1).  Unless  otherwise  mentioned,  when  several  users  are  transmitting 
simultaneously,  it  is  assumed  for  simplicity  that  the  mutual  interference  is  neg¬ 
ligible.  If  this  is  not  the  case,  then  the  same  method  can  be  used,  but  the  rates 
for  any  particular  channel  will  depend  on  the  vector  of  power  allocations. 

Definitions.  The  n-th  scheduling  interval  is  the  real  time  interval  (nA,  nA+A], 
and  the  decisions  for  it  are  made  at  time  nA.  Let  Ln  denote  the  value  of  the 
random  channel  state  process  at  time  nA.  Let  a^n  (resp.,  dl  n)  denote  the 
number  of  packets  arriving  to  (resp.,  transmitted  from)  queue  i  in  scheduling 
interval  n.  The  number  sent  from  queue  i  in  any  scheduling  interval  depends 
only  on  the  channel  and  full  queue  states  at  the  beginning  of  the  interval,  and 
the  control  allocations  to  queue  i  in  the  interval.  Let  Tn  denote  the  minimal 
cr-algebra  that  measures  the  data  (channel  states,  initial  queue  states,  arrivals, 
control  values)  up  to  and  including  time  nA,  and  let  En  denote  the  associated 
conditional  expectation. 

Stability:  Definition.  Owing  to  the  non-Markovianness,  an  appropriate 
definition  of  stability  is  a  “uniform  mean  recurrence  time”  property,  as  follows. 
Suppose  that  there  are  0  <  q0  <  oo  and  a  real- valued  function  F(-)  >0  such 
that  the  following  holds:  For  any  n,  and  <j\  =  minjfc  >  n  :  |cc(AA)  |  <  <70 },  we 
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have3 


En  [^1  A  -F'(x(?T-A))/{|x(nA)|>go}  •  (2.2) 

Then  the  system  is  said  to  be  stable.4  The  definition  implies  recurrence  to 
some  compact  set.  If  the  (absolute  value  of  the  state  of)  the  process  reaches  a 
level  <71  >  qo,  then  the  conditional  expectation  of  the  time  required  to  return 
to  a  value  qo  or  smaller  is  bounded  by  a  function  of  q±,  uniformly  in  the  past 
history  and  in  n.  This  implies  that  the  sequence  {j(nA)}  is  tight  or  bounded 
in  probability  (see,  for  example,  [8,  Theorem  2,  Chapter  6]). 5 

The  basic  Liapunov  function.  For  simplicity  in  the  computations  (and 
because  they  yield  controls  that  are  readily  implemented)  the  basic  Liapunov 
functions  will  be  polynomials  of  the  form6 

V{x)  =  y ^Cj(xj  +  hj)Pi,  (2.3) 


where  hi  >  0,  Ci  >  0,  and  pi  >  2.  The  hi  serve  the  purpose  of  reducing  the 
dependence  of  the  controls  on  the  queue  size  if  the  content  is  small.  Define 
p  =  max.jp,;.  Let  Vx{-)  (with  components  VXi(-))  denote  the  gradient  of  V(-). 

Assumptions.  The  assumptions  are  discussed  after  being  stated.  Controls 
satisfying  (2.4)  are  called  admissible. 

(A2.1)  There  is  a  real  (resp.,  vector7)  valued  function  /(•)  and  sets  Ui  such 
that  such  that  the  controls  satisfy 

fU >  u(x ))  <  £>  >  x)  €  Ui ,  all  j ,  x ,  (2.4) 

where  jl  is  a  real  (resp.,  vector)  valued  constant.  The  set  of  controls  satisfying 
(2.4)  is  compact. 

(A2.2)  The  maximum  number  that  can  be  sent  from  queue  i  over  a  scheduling 
interval  is  bounded  by  the  minimum  of  (a  constant  times  A)  and  X{.  [I.e.,  the 
actual  number  of  packets  sent  can  be  no  greater  than  x%.\  There  are  functions 
#i(-),  with  values  gi(j,  x,  Ui(j,  #)),  and  upper-semicontinuous8  in  Ui  G  Ui  for 
each  value  of  the  other  variables,  such  that  if  x(nA)  =  x  and  Ln  =  j  then  the 

3The  stability  condition  in  [14]  requires  only  that  return  times  be  finite  w.p.l,  but  the 
mean  return  time  might  still  be  infinite.  The  form  (2.2),  based  on  mean  return  times,  seems 
more  appropriate  in  applications. 

4Restricting  attention  to  the  sampling  times  nA  is  unrestrictive  in  view  of  (A2.6). 

5 A  sequence  {An}  is  bounded  in  probability  if  limK^.oo  supn  P{|Xn|  >  ac}  =  0. 

6 Other  polynomial  forms  can  be  used  in  the  development,  provided  that  V(x)  — >  oo  as 
x  — »  oo,  VXi(x)  — >  0  as  Xi  — >  0,  and  VXi(x)  — >  oo  as  X{  — >  oo,  VXi(x)/V(x)  — >  0  as  x  — >  oo, 

and  Enen/\Vx(x)\  — »  0  as  x  — >  oo,  where  en  is  defined  in  (3.5). 

7We  have  the  vector  case  if,  for  example,  both  power  and  time  are  being  constrained  or  in 
the  multiple  antenna  case. 

8 All  that  is  needed  is  that  there  be  a  maximizer  in  (2.6)  that  satisfies  (2.4). 


6 


conditional  (given  the  data  to  nA)  mean  number  of  packets  sent  from  queue  i 
on  (nA,  A  +  A]  can  be  represented  as 


Enditn  =  A  gi(j,x,Ui(j,x)). 


(2.5) 


(A2.3)  There  are  positive  constants  A “,i  <  K,  such  that  the  sums 

OO 

Cln  =  Y.En  [«A-AA“],  i<K, 

l=n 

are  well  defined  and  bounded,  uniformly  in  n. 

(A2.4)  There  are  IT,-  >  0  such  that  Hj  =  1  and  En  —  H;-]  — >  0  fast 

enough  as  l  —  n  — >  oo  so  that  EnI{ Lt=j}  —  IIj|  is  bounded  uniformly  in 

n. 


(A2.5)  Define  the  average  g.i(x,u(x))  =  ^2jIljgi(j,x,Ui(j,x)).  There  are  Cq  > 
0,  Kq  >  0,  and  an  admissible  control  u(x)  =  x);  i,j},  such  that  if  Xi  >  Kq 

then 

A“  -  cji(x,u(x))  <  -c0 

and  g.i(j,x,Ui(j,x)))  does  not  depend  on  x.  9 
(A2.6)  There  is  a  constant  C  such  that  for  each  i 

E„  [aiJ  <  CA,  l  <  pi, 
where  Pi  is  defined  by  (2.3). 

The  control  rule.  Controls  satisfying  (A2.1)  are  called  admissible.  The  actual 
control  is  any  admissible  control  that  maximizes  in 

max  | ^  14,.  „.,„))}  ■  (2.6) 

It  is  assumed  that  there  is  at  least  one  maximizing  admissible  control  Ui(j,x). 

Discussion  of  the  conditions.  Condition  (A2.1)  is  intended  to  cover  con¬ 
straints  such  as  (2.1)  or  where  the  controls  take  values  in  some  finite  set.  The 
/(•)  and  Ut  are  always  defined  by  the  problem  at  hand,  and  there  is  no  need  to 

9The  A'o  is  introduced  in  (A2.5)  only  because  if  the  queue  content  is  smaller  than  the 
maximum  of  what  can  be  transmitted  on  a  scheduling  interval,  then  the  mean  (weighing  with 
the  n, )  output  might  be  too  small  to  assure  the  — co  value.  For  example,  if  a  queue  is  empty, 
then  the  arrivals  will  exceed  the  departures.  Kq  will  be  of  the  order  of  A. 
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specify  them  any  further.  The  only  additional  conditions  on  the  set  of  admissi¬ 
ble  controls  is  that  there  exist  an  admissible  control  satisfying  (A2.5)  and  one 
maximizing  in  (2.6).  For  cases  where  the  control  is  unique  and  readily  found, 
see  the  examples  in  Sections  4  and  5. 

If  control  is  over  power  only  and  the  total  instantaneous  power  must  be  no 
greater  than  fi,  and  can  take  only  some  finite  set  of  values,  then  (2.4)  reduces  to 
(2.1)  and  U,,  is  the  set  of  allowed  discrete  values  for  queue  i.  Condition  (A2.4) 
would  hold  if  the  channel  state  process  were  a  function  of  a  finite-state  ergodic 
Markov  process.  It  need  not  be  Markovian  itself.  Loosely  speaking,  it  would 
hold  if  the  future  becomes  less  and  less  predictable  (given  the  distant  past)  and 
the  conditional  likelihood  of  any  channel  state  approaches  an  average  value. 
Condition  (A2.5)  is  likely  very  close  to  being  necessary.  It  concerns  an  averaged 
system,  where  the  input  is  just  a  fluid  flow  at  the  mean  rate  and  the  output  is 
also  a  fluid  flow,  whose  value  is  the  mean  over  all  channel  states  (i.e.,  there  is 
no  randomness).  If  there  was  no  control  under  which  the  condition  held,  then 
even  this  mean  system  would  be  unstable  and  the  physical  system  might  not  be 
stable. 

Condition  (A2.6)  will  hold  if  the  inputs  occur  continuously  in  time,  but  at 
a  bounded  maximum  rate  (the  usual  case).  Then  the  maximum  input  to  any 
queue  over  a  scheduling  interval  is  proportional  to  A.  Alternatively,  suppose 
that  the  input  rate  is  unbounded  and  a  batch  of  packets  can  arrive  all  at  once. 
Suppose  (without  loss  of  generality)  that  they  arrive  at  instants  fcA,  k  =  1,2,..., 
and  have  bounded  pih  moments.  Let  the  values  over  the  scheduling  intervals 
be  mutually  independent,  and  let  the  probability  of  an  input  to  queue  i  during 
any  interval  be  c,A  (a  discrete  time  Poisson  process).  Then  (A2.6)  holds. 

Comments  on  (A2.3).  Condition  (A2.3)  says  that  the  expectation  of  the 
number  of  packets  arriving  on  any  distant  future  interval  given  the  data  to  the 
remote  past  converges  to  the  average  number  of  arrivals  per  unit  time  as  the 
difference  between  the  times  goes  to  infinity.  The  following  simple  examples 
illustrate  the  condition.  First  suppose  that  the  {ai>n;l,n}  are  mutually  inde¬ 
pendent  with  the  inputs  to  each  queue  being  identically  distributed,  with  A“ 
defined  so  that  EaitH  =  AA“.  Then  the  summands  in  C“n  are  all  zero  and 
C?n  =  0-  ^  the  input  sequence  is  only  M-dependent,  then  all  but  at  most  the 
first  M  summands  in  Cfn  are  zero. 

Let  us  now  examine  C“n  more  closely  to  understand  why  our  requirement  on 
its  convergence  and  boundedness  is  reasonable.  Henceforth  let  us  lump  together 
all  arrivals  in  the  nth  interval  and  suppose  (w.l.o.g)  that  it  occurs  at  the  end. 
Consider  queue  i.  First  suppose  that  arrivals  occur  in  batches  at  (real  time) 
moments  r“n,  i  =  1,2,...,  which  is  assumed  to  be  a  renewal  process  with  values 
in  {nA,  n  =  1, 2, . . .}.  Define  the  mean  interval  =  E[r^n+1  —  r“„],  and  let  the 
number  of  arrivals  at  the  renewal  time  T°n  be  denoted  by  ufn.  For  each  queue 
i,  let  the  ufn,n  <  oo,  be  mutually  independent  and  identically  distributed 
with  mean  vf,  and  suppose  that  the  sequences  for  the  different  queues  are 
mutually  independent.  Define  A“  =  uf  /m“  and  let  A^“n  ;,  /  =  1,2,...,  denote 
the  sequence  of  renewal  times  that  occur  after  time  nA.  By  the  independence 


and  renewal  property,  for  any  integer  v  >  1,  we  have  En[^nv+1  —  J  = 
m“/ A  and,  hence, 


En  y 


Avf 

ai,i  „ 

= 

mi 

r?  A  w? 


A 


=  0. 


Thus,  C°n  reduces  to 


Ctn  = 


l=n 


Ac? 


aiyi 


which  is  just  a  scaled  residual  time  term 

A<„,i  -  nA 


vfEn 


1  - 


where  the  ratio  is  the  time  to  the  next  arrival  divided  by  the  mean  interar¬ 
rival  time.  Thus  (A2.3)  holds.  If  the  amounts  remain  independent,  but  the 
interarrival  times  are  correlated,  then  C'“n  is  just  the  residual  time  term  plus 


V?  Y  En  1 


and  the  condition  holds  under  suitable  mixing-type  conditions  on  the  interarrival 
times.  Analogously,  if  the  independence  of  the  amounts  is  dropped,  then  the 
condition  will  hold  under  suitable  mixing-type  conditions  on  the  interarrival 
times  and  amounts. 


Example.  The  simplest  example  is  where  the  ai,n,i  <  K ,  sequences  are  i.i.d., 
Ln  is  an  ergodic  Markov  chain  and  only  one  queue  can  be  selected  in  any 
interval.  Let  AA“  denote  the  mean  number  of  arrivals  to  queue  i  in  an  interval. 
Then  =  0  in  (A2.3).  In  this  example,  let  Ui(j,x )  denote  the  probability  of 
selecting  queue  i  under  channel  state  j  and  queue  state  x.  Let  A f(j)  denote  the 
maximum  rate  of  transmission  per  unit  time  if  i  is  selected.  Then  we  have  (rate 
per  unit  time  times  A  times  the  probability  of  selecting  i )  A gi(j,x,Ui(j,x))  = 
mm{A\f(j),Xi}ui(j,x).  Suppose  that  there  are  cti(j)  with  JT  cti(j)  =  1  for  all 
i,  such  that  A?  —  JT  <  0.  all  i.  Then  (A2.5)  holds.  This  is  just 

a  requirement  on  a  mean  flow  and  is  analogous  to  the  mean  flow  condition  in 

[14]- 


3  Proof  of  Stability 

The  key  is  to  show  that  if  a  certain  “mean  flow”  system  is  stable,  then  the  actual 
physical  system  is.  Proofs  of  stability  most  often  work  with  Markov  models, 


9 


where  the  Liapunov  functions  are  supermartingales  for  large  values  of  the  state. 
In  our  case,  the  sequence  of  interest  is  {x(nA)},  which  will  not  be  Markov 
unless  the  members  of  the  sequences  of  channel  states  and  arrival  numbers 
are  all  mutually  independent.  In  the  present  case,  these  latter  sequences  are 
not  necessarily  even  Markov.  The  queue  might  grow  “on  the  average”  when 
in  certain  channel  states  and  decrease  “on  the  average”  when  in  others.  All 
that  we  care  about  is  that  in  the  long  term  these  effects  (and  the  effects  of 
the  bursty  non  Markov  arrival  processes)  are  averaged  in  some  way  due  to  the 
random  variations  of  the  channel  and  arrival  processes  so  that  the  behavior  of 
the  physical  system  is  approximated  by  the  “mean  flow  ”  system.  The  idea  of 
a  stochastic  Liapunov  function  that  has  the  supermartingale  property  for  large 
queue  states  is  still  crucial,  but  it  is  obtained  a  little  indirectly.  One  starts  with 
a  Liapunov  function,  which  is  a  function  only  of  the  queue  state,  and  that  can 
be  used  for  the  mean  flow  system.  This  cannot  be  applied  to  the  true  physical 
system  to  get  the  desired  result.  But  it  turns  out  that  that  this  Liapunov 
function  can  be  modified  by  the  addition  of  certain  bounded  terms  that  allow 
averaging  of  the  random  effects  of  the  input  and  channel  processes,  and  the 
modification  will  serve  our  needs.  The  basic  Liapunov  function  is  (2.3).  The 
perturbations  will  be  defined  first,  and  then  the  modified  or  perturbed  Liapunov 
function  will  be  defined.  As  usual  in  stability  proofs  using  Liapunov  functions, 
the  work  consists  essentially  of  a  sequence  of  bounds  on  conditional  expectations 
of  differences  between  of  various  functions. 

The  basic  idea  behind  the  perturbed  Liapunov  function  method  can  be 
loosely  summarized  as  follows.  Suppose  that  EnV(x(nA  +  6))  —  V(x(nA))  = 
cn,  a  random  quantity.  Suppose  that  there  is  a  constant  c  <  0  such  that 
6Vn  =  J2i=nEn[ci  —  c]  is  well  defined.  Define  Vn  =  V(a;(?rA))  +  6Vn.  Then 
En8Vn+1  -  bVn  =  -(c„  -  c)  and  En Vn+\  -  Vn  =  cn  -  [cn  -  c]  =  c  <  0.  Thus,  the 
use  of  the  perturbation  allows  the  replacement  of  the  random  cn  by  a  “mean.” 
This  idea  will  be  used  in  the  proof. 

Theorem  3.1.  Under  (A2.1)-(A2.6)  and  rule  (2.6),  the  system  is  stable. 
Proof.  For  each  i,j,n ,  define 

OO 

Ci,j,n  =  ~J2En  ~  ni]  . 

l=n 

W“n  =  VXi(x(nA))C?n, 

=  A14i  (x(nA))gi (j,  x(nA),  u; (j,  x(nA) ) CfJ  n , 

where  u(-)  is  defined  in  (A2.5),  and  Cfjn  is  well  defined  by  (A2.4).  We  will 
use  the  “perturbed”  (and  time  dependent)  Liapunov  function  Vn  whose  value 
at  time  nA  is 

Vn  =  V{x{nA))  +  £  6V&  +  J2  (3-1) 

i  ij 
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where  U(-)  is  defined  by  (2.3).  Let  u(-)  be  an  admissible  control  that  max¬ 
imizes  in  (2.6).  Since  Xi(nA  +  A)  —  Xi(nA)  =  <Zjin  —  ditU  and  Endi,n  = 
A  JU  g.i(j,  x(nA),Ui(j,  :r(nA)))I{£n=j},  a  first  order  expansion  under  u(-)  yields 


EnV{x(nA  +  A))  -  V(x(nA))  =  ^^(ifiiA))  [En  (ai>n  -  dit„)]  +  e\ 


=  J2Vx*(x(nA^ 


E  9i(j,  x(nA),Ui(j,  x(nA)))I{Ln=j} 


(3.2) 


where  the  “error”  term  ei  is 


exn  =  EnV{x{nA  +  A))  -  U(:r(nA))  -  ^  I4.(a;(nA))  [Enait„  -  Endiin\ . 

i 


By  the  fact  that  u(-)  maximizes  in  (2.6),  for  each  j, 


-  E  Vxi (x(n&))9i(j,  x(nA),Ui(j,  x(nA))) 

i 

<  -  E  VXi(x(nA))9i(j,  x(nA),Ui{j,  x(nA)). 

i 


(3.3) 


Analogously,  we  can  write 

EnbVi  n+ 1  —  EVin  =  —VXi(x(nA))En  [aj>n  —  AA“]  +  ein, 


(3.4) 


where 

ei,n  =  En  [VXi(x(nA  +  A))  -  VXi(x(nA))]  C“n+1. 

Note  that  the  main  term  on  the  right  of  (3.4),  added  to  the  component  VXi  (x(nA))Enaitn 
in  (3.2),  yields  the  “average”  VXi(x(nA))A\1.  Thus  by  adding  the  perturbation 
SV?n  to  V(x ),  we  effectively  replace  the  Enai^n  in  (3.2)  by  the  average  AA“. 

This  is  the  motivation  behind  the  form  chosen  for  the  perturbation.  A  similar 
result  will  be  seen  to  hold  for  the  service  variables. 

If  Xi(nA)  >  K0  +  K i,  where  Ki  is  the  maximum  that  can  be  transmitted  on 
an  interval  and  Kq  is  defined  in  (A2.5),  then,  ^(nA  +  A)  >  Kq  and,  by  (A2.5), 
gi(j,  x(nA),Ui(j,  x(nA)))  =  g,(j,  x(nA  +  A),  Ui(j,  x(nA  +  A))).  Using  this,  we 
have 

EnSV^n+ 1  -  6V&<n 

=  AVXi{x{nA))gi(j,x{nA),Ui(j,x{nA)))  [l{Ln=j}  —  Hj-]  +  ef  jn, 

where 

etj,n  =  AEn  Wxi {x(nA  +  A))  -  VXi(x(A))]gi(j,x(nA),Ui(j,x(nA)))Cfjtn+1. 

Note  that  the  jth  summand  in  the  brackets  in  the  right  hand  term  of  (3.2), 
with  u(-)  replaced  by  u(-)  and  added  to  the  first  term  on  the  right  of  (3.5),  is 
-AVXi(x(nA))gi(j,  x(nA),  Ui(j,  a:(nA)))IT, . 
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Define  6ij  =  ay  —  dij.  By  the  definition  of  V (•)  and  the  binomial  expansion, 


= 


Pi 

E«E 

i  1=2 


Pi 


(xi(nA)  +  hi) 


Pi~ l  T?  ol 


where  the  terms  with  the  large  parentheses  are  the  binomial  coefficients.  The 
terms  e“n  and  efjn  are  bounded  by  a  constant  times  the  absolute  value  of 


E„ 


(xi(nA)  +  hi  +  Sitn)p'  1  -  (xi(nA)  +  hi) 


\Pi~l 


Substitute  the  inequality  (3.3)  into  (3.2).  Then  combining  the  above  computa¬ 
tions  and  canceling  terms  where  possible  yields 

EnYn+l  —  Vn  <  0(A)  +  en  + 


E  AVXi(x(n  A)) 

i:Xi(nA)'>Ko~\-Ki 


E  x{nA),Ui(j,  ®(«A)))rij  , 

3 


(3.6) 


where  the  0(A)  term  is  due  to  the  components  i  for  which  Xj(nA)  <  Kq  +  Ad, 
and  is  bounded  in  x,  and 


en  =  en  +  Ee^  +  EebJ>  =  0(l)AE(Xi(nA)  +  /li)Pi  2-  (3-7) 

i  i,j  i 

Note  that,  by  (A2.5),  [A“  -  9i(j,x{nA),ui(j,x(nA)))Hj\  <  -c0  for  all 
i  such  that  Xi(nA)  >  Kq.  Thus  under  any  control  u(-)  determined  by  the 
maximization  rule  (2.6),  we  must  have 


EnVn+  i-Vn<-  ^2  AVXi(x(nA))c0  +  0(A)  +  en.  (3.8) 

i\Xi  (nA)'>Ko~\-Ki 


The  dependence  of  en  on  X{(nA)  is  dominated  by  a  constant  times  Axp~  (nA). 
The  main  term  on  the  right  of  (3.8)  is  of  the  order  of  Ei-ii(nA)>K0+A’i  :r?_1(nA). 
Hence,  there  is  a  constant  K2  such  that  if  |x(?iA)|  >  K2,  then  we  can  write 

EnVn+ i  ~Vn<  -A  ^2  VXi(x(nA))c0/2 

i:Xi(nA)>K0  +  I<i  /„ 

= -A  ^2  pi{xi(nA)  +  hl)p,~1c0/2. 

i:Xi(nA)'>K.Q-\-K\ 

In  particular, 

EnVn+i  —Vn—*  —oo,  uniformly  in  n  as  x(nA)  — >  oo.  (3.10) 

Before  continuing,  let  us  note  the  following.  Suppose  that  x(nA)  were 
Markov,  and  there  were  constants  Ci  >  0,AT3  >  0,  such  that  EnV(x(nA  + 
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A)  —  V(x(nA))  <  —Ci  if  V(x(nA))  >  K3.  Then  the  segments  of  the  process 
V(x(nA))  on  the  time  intervals  on  which  the  V(x(nA))  are  no  less  than  K3  are 
supermartingales  and  classical  stability  theorems  [7]  imply  that  Ext  <  V(x)/c\, 
where  r  is  the  return  time  to  the  set  where  V  ( x )  <  K3  and  Ex  is  the  expectation 
given  initial  condition  x.  This  is  (2.2)  in  this  special  case. 

In  the  general  case  of  interest  in  the  theorem,  x(nA)  is  not  necessarily 
Markov  and  the  Liapunov  function  is  (3.1).  The  result  (3.10)  implies  that  the 
segments  of  the  process  Vn  on  the  time  intervals  on  which  the  |x(nA)|  are  no  less 
than  A'2  are  supermartingales.  Obtaining  the  bounds  on  the  conditional  mean 
return  time  from  (3.10)  is  similar  to  what  is  done  in  the  Markov  case.  One 
just  has  to  ensure  that  the  effects  of  the  perturbations  can  be  appropriately 
“dominated.”  Using  (A2.3)  and  (A2.4),  we  have  the  bound 


0(1)  E  (xi(n A)  +  hj)Pi  1, 

i 


(3.11) 


which  is  of  smaller  order  than  U(-).  Hence,  Vn  is  bounded  below  and  goes  to 
infinity  (uniformly  in  n)  as  x(nA)  — >  00.  This  and  (3.10)  imply  the  conditions 
of  [8,  Theorem  2,  Chapter  6]  which  yields  (2.2),  and  we  fill  in  the  details  next. 

By  (3.10),  there  are  c\  >  0,<7o  >  0,  such  that,  for  |x(nA)|  >  q0,  EnVn+i  — 
Vn  <  —ci.  Given  small  6  >  0,  (3.11)  implies  that  for  q0  sufficiently  large, 
\V(x(nA))  —  Vn\  <  <5(l+V(x(nA)).  Let  tr0  be  astopping  time  for  which  |a?(cr0) |  = 
C2  >  qo ,  and  define  the  stopping  time  =  minjnA  >  cto  :  |x(nA)|  <  <70}.  Then 
we  have 

E„ 0Vai  -  Vao  <  —CiEao  [a  1  -  Co] .  (3-12) 

Using  (3.12)  and  the  bound  (3.11)  on  Vn  —  V(x(nA))  to  bound  Vai  —  U(x(cr.j)), 
i  =  0,1,  yields 


-6Eao[l  +  V(x(ai))]  +  ECJoV{x{ai)) 

<  EaoVai  <  -ciE^d!  -  cr0)  +  [<5  +  U(x(cr0))(l  +  5)] 
or 

E  (ai  _  ao)  <  +  l/(cc(go))(l  +  £)  +  ^Eq0V{x{(Ti)) 

Cl 

which  is  equivalent  to  (2.2),  since  V(x(ai))  <  sup|xi<90  V(x).  ■ 


4  Examples 

The  examples  are  only  idealized  outlines  of  a  sampling  of  the  possibilities.  In 
all  cases,  we  write  x(nA)  =  x.  Let  the  bit  interval  for  queue  i  be  be  A*.  The 
data  arrival  process  need  not  be  specified  in  the  following  discussions. 

Example  4.0.  The  simplest  example  is  where  only  one  queue  can  be  powered  in 
any  interval,  as  follows.  The  constraint  is  (2.1)  and  in  (A2.1),  Ui  =  {0,  u}.  There 
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are  A;(j)  >  0  such  that  min{A i{j),Xi}  is  the  number  of  packets  transmitted  in 
the  interval  if  queue  i  is  selected.  Let  Ii(j,  x)  denote  the  indicator  function  of 
the  event  that  queue  i  is  selected.  Then  <?,(•)  is  defined  by  Agi(j,x,Ui(j,x))  = 
min{Aj(j), a;).  To  maximize  the  expression  (2.6),  we  choose  the  i  that 
gives  argmax{T4.(a;)  min{Aj(j),  a;,;}}. 

Example  4.1.  [Power  only  control.]  Suppose  that  the  queued  data  is  optimally 
recoded  before  transmission  and  that  the  queues  and  outputs  are  measured  in 
terms  in  bits.  The  Shannon  capacity  of  a  white-Gaussian  channel  with  un¬ 
restricted  alphabet  and  fixed  symbol  intervals  is  log(l  +  S/N)/ 2  per  symbol, 
where  S  and  N  are  the  signal  and  noise  powers,  resp.,  at  the  receiver  [6,  Eqn. 
10.17].  [For  complex  signals  multiply  by  2.]  Suppose  that  this  form  models 
all  the  channels  and  there  is  negligible  mutual  interference.  Let  Ki  denote  the 
number  of  symbols  that  can  be  transmitted  per  second  for  queue  i.  Since  the 
symbol  intervals  are  assumed  to  be  very  short  relative  to  the  scheduling  interval, 
with  appropriate  coding  the  total  number  of  bits  transmitted  over  a  scheduling 
interval  is  well  approximated  by  AKi  times  the  capacity.  The  power  constraint 
is  M  for  all  jjX,  where  jl  is  the  total  available  power  and  Ui(j,x) 

is  the  actual  power  applied  to  transmission  from  queue  i  in  the  interval.  This 
constraint  is  (A2.1)  in  this  case,  where  the  C/j  can  be  taken  to  be  the  nonneg¬ 
ative  real  numbers  [0,  oo).  Then,  there  are  constants  such  that  if  the 

channel  state  is  fixed  at  j,  the  channel  capacity  Ci(j,Ui(j,x))  for  queue  i  under 
Ui(j,x)  is 

CM,  Ui(j,  x))  =  -  log  [1  +  aiijUi (j,  x)/Nij\ .  (4.1) 

The  a,;j  scales  for  channel  attenuation.  Then  we  can  suppose  that,  when  the 
channel  state  is  j,  di>n  =  A gi(j,x,u,i(j,x))  =  min  {A Ci(j,Ui(j,x)),Xi}.  This 
defines  the  functions  gi(-).  Let  V(x)  =  J2ixi^  f°r  some  p  >  1.  Then,  when  the 
channel  state  is  j,  the  control  maximizes 

T.  zf-1  min  |  A^  log  (1  +  aijUi(j,  x)/Ni:j)  ,xA  .  (4.2) 

Due  to  the  strict  concavity  of  the  function  log(l  +  cu),  several  queues  might  be 
powered  simultaneously.  If  desired,  one  can  add  the  constraint  that  only  one 
queue  is  powered  during  any  scheduling  interval.  In  this  case  one  looks  for  the 
summand  in  (4.2)  that  is  largest  when  Ui(j,x)  =  u,  and  all  power  goes  to  that 
user.  If  the  maximizer  is  not  unique,  then  select  any  one,  or  randomize  among 
them. 

Example  4.2.  [Power  only  control.]  Now  suppose  that  the  number  transmit¬ 
ted  is  proportional  to  the  power,  and  one  might  transmit  from  several  queues 
simultaneously  with  negligible  mutual  interference.  In  particular,  let  there  be 
constants  A f(j)  such  that  the  number  of  packets  per  unit  time  that  can  be  sent 
from  queue  i  under  channel  state  j  is  A f(j)ui(j,x)  for  m(j,x)  £  Ui ,  a  finite 
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set  for  each  i.  We  still  require  the  constraint  (2.1).  The  function  gt ( • )  is  de¬ 
fined  by  Agi(j,x,Ui(j,x))  =  min  x),  a;*}  .  Given  V(x)  of  the  form 

in  Example  4.1,  the  maximization  is  trivial. 

There  are  many  ways  that  one  can  realize  the  number  transmitted  being 
proportional  to  power.  If  bandwidth  is  the  single  resource  to  be  allocated  and 
power  is  proportional  to  the  bandwidth,  then  it  is  often  the  case  that  the  rate  is 
proportional  to  power.  With  CDMA,  one  possibility  is  to  allow  several  “spread¬ 
ing  sequences”  per  queue  [15].  Another  possibility  is  to  allow  the  bit  interval 
(duration)  to  be  controllable.  In  particular  suppose  that  (for  queue  i)  there  are 
basic  time  intervals  of  which  the  actual  bit  interval  A\  is  one  of  the  multi¬ 
ples  m8\,m  =  1, .. .  ,TOj,  for  some  given  to*  <  oo.  Suppose  that  the  channels 
are  white-Gaussian,  that  there  is  perfect  synchronization  at  the  receiver,  and 
that  the  system  design  is  such  that  if  the  power  allocation  to  any  queue  is  not 
zero  then  it  must  be  such  that  the  SNR  per  bit  at  the  receiver  is  at  least  some 
given  positive  number  Ci(j).  This  minimal  amount  Cj(j),  together  with  the  bit 
interval,  determines  the  capacity.  Thus,  cutting  the  bit  interval  in  half  would 
require  twice  the  power  for  the  same  SNR.  The  chosen  bit  interval  is  constant 
during  a  scheduling  interval. 

Example  4.3.  [Power  and  time  control.]  In  Examples  4.1  and  4.2,  the  total 
power  was  constrained  to  p  at  each  time  and  the  transmissions  from  all  selected 
queues  in  an  interval  were  done  simultaneously.  Alternatively,  suppose  that  we 
schedule  by  dividing  time  as  well  as  power  in  the  following  generalized  form  of 
TDMA.  For  each  queue  i,  channel  state  j,  and  queue  state  vector  x,  partition  the 
nth  scheduling  interval  into  K  subintervals  (some  subintervals  might  have  length 
zero)  of  total  lengths  Apj,  x),  i  <  AT,  where  JA  A i(j,x)  =  A.  Summarizing, 
in  the  nth  scheduling  interval,  we  transmit  from  queue  i  on  a  total  time  Aj>n, 
which  can  depend  on  i  and  on  the  current  queue  and  channel  states  x,j. 

Suppose  that  we  can  vary  the  power  within  an  interval  provided  that  it 
averages  to  p  over  the  interval.  In  particular,  let  Pi(j,x)  denote  the  power 
applied  to  queue  i  during  the  part  of  the  interval  that  queue  i  is  being  worked 
on.  Then  the  full  vector-valued  control  Ui(j,  x)  is  (pi(j,  x),  A;(j,  x)).  Then,  in 
lieu  of  the  pointwise  constraint  (2.1),  we  now  constrain  average  total  power  over 
the  interval  and  the  constraint  (2.1)  is  replaced  by 

^2PiU,x)[Ai(j,x)/A]<p,  all  j.  (4.3) 

i 

Now  consider  the  setup  in  Example  4.1,  but  where  only  one  queue  can  be 
powered  at  a  time  and  the  scheduling  interval  is  divided  among  the  queues,  as 
above.  Define  gi(-)  by 

f  Kj, 

A gi(j,  x,  Ui(j,  x))  =  min  <  A i(j,  x)  -A  log  [1  +  oiijPi{j ,  x)/Nij] ,  xt 
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Then,  the  rule  (2.6)  is  the  maximizing  (power  and  time)  value  in 
max  {e  VXi(x)gi(j,x,Ui(j,x)) | , 
with  constraints  (4.3)  and  JA  A,(j,  a;)  =  A. 

Example  4.4.  The  general  scheme  is  not  restricted  to  systems  with  only  one 
antenna.  This  will  be  briefly  illustrated  when  there  are  two  antennas,  and  we 
have  power  only  control,  with  one  queue  being  scheduled  at  a  time,  analogously 
to  the  situation  of  Example  4.0.  For  notational  simplicity,  suppose  that  there  are 
two  antennas.  Let  the  channel  state  for  antenna  a  be  denoted  by  ja ,  write  the 
full  channel  state  as  j  =  (j1,  j2),  and  let  uf(j ,  x )  denote  the  power  that  antenna 
a  applies  to  queue  i.  The  total  power  constraints  are  JA  u“(j,  x)  =  ua,  a  =  1,  2. 
The  Ut  in  (A2.1)  becomes  a  product  set  U }  x  U2,  where  £/“  =  {0,u“}. 

The  scheduling  can  be  done  independently  between  the  antennas,  or  there 
could  be  coordination.  We  will  consider  all  possibilities.  First  allow  the  two  an¬ 
tennas  to  be  scheduled  independently.  One  simply  uses  (2.6)  for  each  antenna 
with  uf(j,x)  depending  only  on  (i,ja,x).  Let  min{A“(j), x{\  denote  the  num¬ 
ber  of  packets  transmitted  in  an  interval  on  antenna  a  when  queue  i  is  selected 
for  it  and  the  (channel,  queue)  state  is  ( j,x ).  When  the  antennas  are  scheduled 
independently,  there  are  two  functions  g?(ja,x,uf(ja,x)),  a  =  1,2,  each  ob¬ 
tained  analogously  to  the  construction  in  Example  4.0.  Now  let  us  coordinate 
the  decisions.  The  simplest  form  of  coordination  is  to  modify  the  foregoing  by 
allowing  only  one  queue  to  be  powered  at  a  time  at  each  antenna.  Let  If  (j,  x) 
denote  the  indicator  of  the  event  that  queue  i  is  selected  by  antenna  a.  Then 
the  constraints  can  be  considered  to  be 

YJIi{j^)  =  l,a  =  1,2,  ^/f(j,z)<  1.  (4.4) 

i  cx. 

The  expression  (2.6)  becomes 

max  E  Vxi  (*^)  {min  {  (j) ,  Xi  {  1^  {j*  x  j  T  min  {  A^  (j) ,  x i  {  I ^  (j ■  x') }  .  (4-5) 

i 

Another  option  allows  the  possibility  of  using  a  more  sophisticated  coding 
scheme,  say  that  of  space  time  coding  [13],  still  under  the  constraints  (4.4). 
In  this  case,  both  antennas  are  used  for  the  selected  queue,  with  the  coding 
accounting  for  the  fact  that  there  are  two  antennas.  This  gives  a  different 
function  g(-).  There  might  be  some  channel  and  queue  states  for  which  the 
value  of  (4.5)  is  larger  than  the  value  of  (2.6)  under  space-time  coding  and 
conversely.  One  would  take  the  best  choice. 
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5  Examples  of  Controls  and  Relation  to  Optimal 
Controls 

Consider  Example  4.2,  where  the  rate  of  transmission  is  proportional  to  power 
(the  model  used  in  [1])  and  let  V(x)  =  J2ixi+1>P  —  T  Then  the  control  given 
by  (2.6)  is  the  maximizer  in 

max  A  i  (, j)m  {j,  x) 

over  the  allowed  discrete  values.  In  the  rest  of  the  discussion  of  the  controls,  we 
ignore  the  small  queues,  which  are  readily  accounted  for. 

A  two-dimensional  case.  Consider  a  two-queue  and  two-channel  state  ex¬ 
ample  with  no  constraints  on  the  power  allocation.  Let  Af(l)  >  Aff(l),  and 
Af(2)  <  A(j(2).  For  channel  state  1,  apply  all  power  to  queue  1  when 

x1/x2>[\d2(l)/\d1(l)]1/p.  (5.1) 

For  channel  state  2,  apply  all  power  to  queue  2  when 

^i>[A?(2)/A^(2)]1/p.  (5.2) 

As  p  — >  oo,  the  switching  lines  move  to  the  diagonal. 

An  alternative  Liapunov  function.  Now  consider  the  Liapunov  function 
V(x)  =  +  hi]p+1.  With  equal  hi,  this  gives  results  that  are  closer  to  the 

numerically  computed  optimal  controls  for  an  approximating  diffusion  with  cost 
rate  max{xi,a;2}  (see  below).  The  inequalities  (5.1)  and  (5.2)  are  replaced  by. 
respectively, 


xi  +  hi/x2  +  h%  >  [Ag(l)/A?(l)]  ^  , 

(5.3) 

z2  +  h2/x i  +  h!>  [A?(2)/Ag(2)] l/p  . 

(5.4) 

Here  the  cost  rate  is  less  sensitive  to  differences  in  the  Xi  unless  they  are  large. 

Comparison  with  optimal  controls.  It  is  interesting  to  relate  the  control 
forms  determined  by  ((5.1),  (5.2))  and  by  ((5.3),  (5.4))  to  optimal  controls  for 
diffusion  processes  with  similar  dynamics.  Figure  5.1  gives  the  optimal  switching 
surfaces  for  control  problem  for  the  diffusion  process  dx  =  g(x,u(x))dt  +  adw. 
The  qualitative  form  of  the  controls  did  not  depend  on  a.  We  let  Af(l)  > 
A2(l),Af(2)  <  A2(2),  and  the  optimal  controls  are  obtained  by  numerical  so¬ 
lution  using  the  methods  of  [9].  The  cost  rate  is  k(x)  =  maxjii,^}  for  the 
left-hand  figure  and  '^2iXi  for  the  right-hand  figure.  The  total  cost  was  ei¬ 
ther  of  the  ergodic  or  discounted  form  with  small  discount  factor,  with  little 
difference  in  the  controls  between  them.  Similar  results  hold  for  discrete  time 
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problems.  The  slopes  for  the  linear  cost  rate  tend  to  be  larger  than  those  for 
the  max{:ci, £2}  criterion.  Equivalently,  for  the  linear  cost  rate  we  put  more 
emphasis  on  using  the  most  efficient  channel  and  less  on  attaining  closeness  of 
the  queues.  Note  the  similarity  of  the  controls  to  the  forms  given  by  ((5.1), 
(5.2))  and  ((5.3),  (5.4)).  The  controls  in  the  left  hand  figure  have  the  form  of 
((5.3),  (5.4)),  and  those  in  the  right  hand  figure  have  the  form  of  ((5.1),  (5.2)). 
Thus,  with  appropriate  choices  of  the  parameters,  the  controls  are  close  to  those 
for  optimal  processes  for  closely  related  problems. 


Figure  5.1.  Switching  curves  for  the  optimal  control: 
k(x)  =  max{:r  1,0:2}  and  JUxj. 
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